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Sommary 



This pmjcd developed computer-based language leaching software to assist beginning second language Icamcis 
develop listening compreliension skills. Students interact with oon^ter-based simulations of world 
problcnB, wKch require their undcistanding of oral language provided by the computer systcna. The project 
combined two independent insghts. Rist, it embraced an approach to language learning and teaching called the 
commuwc^tive appnxK* (along with a more narrowly defined sibling, the oojqp/eAcas/oD Mppio^c^ which 
emphasize placing language learning in a task-criented, problem-solving, communicative context Second, it 
embraced an approach to computer tutor design called 'programming by rehearsal*, in whidi teadiers titilize a 
theatrical metaphor (teacher as diiector) to develop novel simulati<ms within the oljccl-crienled authoring 
ccxnpoosni of the computer tutor systcna The project developed software to produce and operate such interactive 
language learning amulations and associated materials to assist language teachers in developing 
communicatively oriented simulations. 



Beginning Second Language Instruction 
Computer-Based Curriculum Improvements 
FIPSE Grant: G00854I129 

Russell S. Tomlin 
Sarah A. Douglas 
University of Oregon 

Executive Summary 



A. Project Overview 

This FIPSE project was directed at improvements in undergraduate foreign language education at the University 
of Oregon (and similar institutions aaoss the U.S.). More specifically, the projea was directed at improving 
foreign language instniction for beginning level students and in the specific area of listening comprehension. 
At project conception, we believed that a new kind of computer-based support system could be created which 
would permit students to engage in listening oriented communicative interactions with simulations of real world 
problems. 

The principal outcomes of the project were: (1) user firiendly software for use by teachers in designing and by 
students in interacting with language teaching simulations, and (2) new insight into the nature of language 
learning and teaching. Both of these should have some impact on foreign language instruction and ot the design 
of other software systems. 



B. Purpose 

Oral communication ridUs— the ability to comprehend and prcxluce oral discourse— are crucial in nearly every 
educational, business, and scientific setting of language use. Yet the development of oral communication skills 
remains a difficult thecwetical and practical problem, and traditional language teaching approaches regularly fail 
to help many Icameis, The central problem addressed by this projea was to design a computer assisted language 
instruction system v/bkh could help brg^nning language learners develq? their aural comprehension abilities. 
The reasons for targeting beginners as well as listening comprehension were three. First, relatively little 
software is directed M true beginners. Second, relatively little software is directed at improving listening 
comprehension. Third, the computer environment is best suited to work in teaching listening. 

C. Background and Origins 
Origins 

Given an interest among our graduate students in computer-assisted language instruction and our examination of 
available systems, we believed existing sc^are and the approaches to language teaching taken under them to be 
generally unsatisfactory, both pedagogk:aIly and computationally. From a pedagogical viewpoint, existing 
CALL (computer assisted language l^nin^ software did not incorporate innovations in seccmd language 
leaching approaches otherwise important to die field, in particular task-oriented, probIem*solving approaches 
subsumed under the general commurdcative approach. From a computational viewpoint, existing CALL 
utilized very simpli^ programming strategies and control structures while not permitting easily truly creative 
teacher initiated variation in student lessons. 

Institutional Context 

The narrow target of our project was undergraduate students beginning foreign language study at the University 
of Oregon, a populackxi of some 2100 students. These students were distributed in several languages 
departments: Romance, Germanic, Russian, East Asian (Japanese and Mandarin), and the AB. 
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In addition, we had resources available to us througji the American English Institute (AEI). The AEl represents a 
rather unique context for this FIPSE project Unlike most English language teaching units on majw university 
campuses, the AEI, in its operational charter specifies that it must support and promulgate research in second 
language learning and teaching. Thus, in addition to the direct grant support by FIPSE, the AEI provided 
substantial material siqpport to the project in the form of new equipment and additional personnel support 



D* Projeci Description 

Ovcxall, the principal goal of this project was to develop a second language leaching software systcni to assist 
nil proficiency learners develop beginning listening comprehension skills. We entered the project with a 
number of elaborated examples of the kind of system we wanted to produce, but without great certainty about 
the precise path that would gel us there. In some respects, one of the most important outcomes of the project, 
besides the actual s(^tware developed, was the development of understanding of wJiat some of the basic issues are 
in language learning and teaching that must be faced in software development as well as what some of the basic 
issues are in software design and implementition. As we proceeded, and even as we continue now, we redefined 
our project goals to pursue some fundament^ questions in this learning/teaching domain that conventional 
wisdom did not address. 

Language Learning and Language Teaching 

From the point of view of language teaching theory our project draws on two important and innovative 
approaches to second language learning and teaching: the communicative and the comprehension approaches. 
Proponents of the communicative approach argue that successful language learning occurs when the student is 
provided the oppornmity to solve non-language problems using the developing second language (Widdowson, 
1978; Kiashen and Terrell, 1983). They criticize traditional language teaching for focusing too much effort on 
the conscious discussion and manipulation of rules of language usage and not enough effort on the acquisition of 
the second language grammar through efforts to use that grammar to solve actual communication problems. 
This philosophy int^tes well with the general spirit of ICAI wherein learning is a problem-solving process. 

Proponents of the comprehension approach argue thai second language learning is enhanced when beginning 
stages of language leanung are devoted to developing the ability to understand the second language. Obligatory 
oral production is delayed until the student is able to understand easily utterances in the second language. 
Delaying production improves student performance in other aspects of language acquisition (Posiovsky. 1974, 
1976; Asher, 1966. 1969. 1972. 1974, 1981; Winitz, 1981). 

Our project embraces both of these complementary approaches to language learning and teaching. The 
instructional system we have created involves the student in solving communicative problems interactively with 
the system. The stiklent participates in problem-solving simulations which allow manipulation of objects in a 
physical scenario or j mcroworld. Information about the problem to be solved as well as information about the 
microworld is given in the second language. Meta-level commentary by the tutor is also in the second language. 
The teaching intervention in these simulations can vary fiom highly directed to coaching to purely student* 
controlled exploratioD. 

Design Approach 

Our pedagogical model requires many small problem-solving environments to be built, as well as many expert 
tutCTS. This has motivated us to study the possibility of building a high-level authoring system. Our general 
philosophy in constructing this system is that we want the micnoworlds to be very knowledge-intensive and 
totally integrated wSh the interface. We also want them to be reusable. These two themes have pushed us to 
envision a sort of libfary of miaroworlds and tutoring components. LingWorlds offers the teacher a rather large 
amount of programming power, if the teacher wants to use it, while pennitting teachers with less exp;.rience the 
facility to build single simulations. 

Authoring lessons for computer-based second-language instruction is, traditionally, a time-consuming and 
intensive task whkft has all of the problems of interface construction. With the advent of bit-mapped displays 
and mice, the author is faced with ever increasing design complexity as graphics and sound supplement text 
displays, windows aDow multiple contexts for user tasks, and pointing devices join keyboards. On sophisticated 
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systems such as Inlcrli^ on the Xerox 1 lOO's. the interface programming effort has been informally estimated 
to consume about 80% of the total programming time. The long delay experienced in software development for 
even less complex machines such as the Macintosh can similariy be attributed to the complexity of composing 
over 600 ROM-based interface functions into usable interactive programs. 

For later purposes of building the authoring system as well as generalizing microworlds, we art committed to 
three design methodologies: nq)id prototyping, taxonomic classification, and direa manipulation for the 
interfaces. Wc feel that these methods pro\ide greater programming productivi^ through the ability to custom- 
tailor existing code by ^)ecializing subclasses and adding instances, and by immediate simulation of of code 
modules which shortens the generatc-and-tcst loop. Tliese three approaches are manifested in object-oriented 
programming envinMimcnts like Smalltalk and Lisp with Flavors. Previous work comparing rule-based with 
object-oriented ICAI tutors (Douglas, 1986) suggested that there are significant advantages possible with an 
object-oriented implementation, not the least of which is the accommodation of event-driven, student-initiated 
control with event-driven, tutor-initiated control. Thus, our system, like Programming by Rehearsal, ARK. 
Thinglab, and STEAMER, is almost entirely object-oriented 



AN EXAMPLE: PROVISIONING THE UFEBOAT 

In order to provide the reader with a concrete idea of how the system works, the following is an extended 
example of one microworld called Provisioning the Lifeboat. In this simulation the student is faced with the 
non-linguistic task of provisioning a lifeboat before an ocean liner sinks. Hie simulation begins with an 
introductory animation and instmctions. Tlie computer displays an animation showing an ocean liner 
approaching an iceberg. A simple oral narrative accompanies the animation. As the ship collides with the 
iceberg, a second, ctose-up animation occurs showing the sinking ship. Finally, an on-deck scene of equipment 
and people near a lifeboat is presented. At this point in the simulation, student interaction begins. Oral language 
directs the student to kxations of provisions and equipment needed to use the lifeboat successfully. The student 
responds to the instructions by pointing, clicking or draggmg with a mouse. Only through successful 
comprehension of the spoken language will the student be able to complete the tasks required to use the lifeboat 

While commands can be relatively simple to bepn (TPut the anchor in the lifeboat"), they can become 
increasingly complex (Tut the binoculars in the basket in the boat and then put the waterjug beside them."). 
Note that in this second problem the student must solve several complicated problems: know the names of the 
objects, perform the task in the correct temporal order, select which of the two baskets is the correct k)cation, 
and understand prepositions of location (beside, in» etc.). 

There are three veraons of the Lifeboat simulation whk:h vary the ma of student and tutor initiation and 
control complexity: an explcHaiory (student-initiated), a game, and a directed tutor. They demonstrate the kind of 
versatility that wc want to provide for microworld control. In the exploratory mode, fhc student can use the 
mouse to single-click 6bjects and hear their names (TTie basket'O or double-cUck objects to bear a linguistic 
description of their locatkxis in relation to another object CThe woman is not in the lifeboat*^ The student is 
also fite to drag objects fiom one location to another to hear the cffea of a changed location. Help instructions 
are available by clicking a question mark (?) icon. In the game mode, oral commands dueci the student to 
locations of provisions and equipment needed to provision the lifeboat successfully. The sinking of the ship 
provides lime pressure for completion of the tasks. The system kcqw a score of successfiil tasks. 

Finally, in the iutored lifeboat mode, the directed tutor mode supplements the oral commands of the game 
mode with remedial inlcrvention. The control strategy that we use is derived from protocol studies of a human 
expert tutor. The tutor maintains a curriculum of concepts to be taught and a differential student model to 
determine state transtion information and to diagnose types of errors. If the student fiuls to move any object 
after a few seconds, the command is rq)eated A second such failure causes repetition of the instructions for the 
overall task; after the rtiird failure the system demonstrates the action. If the student moves an object to the 
wrong place, it b returned (by the system) to its initial location and the command is repeated. After several 
failures, the system wiD move the object to the appropriate k)cation, demonstrating the task. The student is 
then given another opportunity. 

These examples devebp the comprehension-based grammar of the nil-proficiency learner in at least the 
following ways: 
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• Development of kxicon/vocabulary. 

• Devdopment of the grammar of ^tial relations. 

• Development of the grammar of reference. 

• Development of basic syntax and word order. 

• Development of the English article system. 

From the point of view of the student, we believe that the system we have developed will be challenging and 
interesting. The student cannot accomplish the needed tasks simply through knowledge of the world, but must 
comprehend the second language utterances. That is, the student is engaged in true communicative behavior, 
using the developing second language to solve meaningful, non-language pr6blcms~the essence of Ae 
communicative j^proach. The student demonstrates mastery of the linguistic task by physical manipulation of 
the world rather than by linguistic production. Thus, flie student works on aural comprehension, die essence of 
the comprehension appn»ch to second language learning. 



E. Project Results 
Pedagogical Efforts 

Complementary to Uie general problem of developing the computer :utor are a number of pedagogical issues 
pursued during the project (1) the nature of tutoring, (2) the nature of language teaching simulations, (3) 
specific curriculum and simulation lessons to be implemented when the system was ready, and (4) general 
principles for developirig listening abilities at beginning levels. 

1. The nature ofiutorimg. Perhaps Uie most fundamental question for this project was posed early on: 
what is it that tutors teach and how do they decide what to do at any given point in a tutorial interaction? 

In our eEforts,we developed an zpTptosch which yielded understanding of what it is tutors do when they teach. 
Using a limited semantic domain, we videotaped expert language tutors engaged in oneon-one Uiiorials with nil 
proficiency learners of various languages. These observations were taken both of face-to-face tutorials arul of 
computer mediated tutorials (ttitorials where, like the computer, the tutor did not have access to visual 
information on the movements of learner face and hands and eyes). From Uiese observations, we have been able 
10 extract significant components and principles of second language tutorials (Douglas, in press; Tomlin, 
Douglas ct al., 1988). 

2. The development •/ model language teaching simulations. During the course of the project 
we developed a number of listening onenled language teaching simulations. They rqwesenl the kinds of 
simulations we believe will prove eflective pedago^cally arvl engaging to the language learner. 

(1) The lifeboat. In this simulation the student must provision a lifeboat in response to oral instructions in 
a second language (either ESL or J^)anese). The student moves objects about the computer screen and places 
them in requested positions in the lifeboat. In order to do this successfully, the siudent must comprehend lexical 
expressions for the various items and relational expressions for the targeted locations. 

(2) Maptiles. This simulation represents one ofour ideal lesson environments. In the development of 
listening abilities it is ipite common to use maps to represent a territory through which the student must 
navigate in ttsponst to iqnit in the target language. The maptiles environment provides a toolbox with which 
leacheis can constnict tfieb own mdps and language lessons based on them. 

(3) The animated dictionary. The animated dictionary (AD) is a lexical support system for students and 
teachers. Students use the animated dictionary to search for kxical items, related items and collocations, a 
variations froni semantic prototypes. Teachers control the extent of student access in a given scenario. 

(4) FlatLand. In this simulation, the student buUds simple two-dimensional configurations from a limited set 
of objects: black or while, large or small, circles or squares. 



(5) Mystery world. This simulation places the student in the position of a witness to some set of events. 
The events aie presented as an animated film of some kind or other. Associated with tiie filni is a descriptive 
sound track which relates the actions witnessed by the student 

3. The nature of simulations. In order to assist teacher in the design of engaging and effective language 
leaching simulations, we examined the underlying organization of task-orientcd^guage teaching simulations 
and developed a taxonomy of simulation problem ty^fcs. 

4. General principles of communicative language teaching targeting listening 
comprehension. We effected a search of the pedagogical and theoretical literature in second language leammg 
and teaching and we interviewed practicing second language teachers in order to create an inventwy of listening 
activities. The inventory of activities is organized to reveal: (1) the necessary prerequisite skills required to 
perform tiie task, (2) the Icaming goals for the task, (3) the expected outcomes for the task, (4) the 
means/methods of presenting/executing the task, and (5) means of evaluating the effectiveness of the task. 

The LingWorlds Simulation System: Implementation 

The LingWorlds tutoring system consists of two component parts. One component is seen and used by the 
learner to engage in language learning simulations. This component we will call the tutorial system. The 
second component is used by language teachers to create simulations and language learning problems. We will 
call this component the authoring system. 

The LingWorlds Tutorial System 

The LingWorlds microworld consists of sequences of scenes of objects. Each object has a graphic display and is 
capable of perfomiing various programmable actions such as spciJdng and moving in response to 1) student 
interactions such as clicking or dragging the object, 2) actions directed by other objects and 3) internally 
generated tutor actions. The language generator contains special routines which manipulate digitized data and are 
written in assembly language to be as efficient as possible. The language generator has a digitized lexicon, and 
a case-fiame semantic grammar, and is capable of dynamically generating utterances fiom a concq)tual 
representation. It is much too primitive to be considered a full-fledged natural language generator but is 
sufficient for our simple language teaching. A sqjarate generator is needed for each language taugjit 

LingWorlds microworlds are generated by the authoring system (see below). Each object is essentially an 
object-oriented programming construct, as is the tutor component If the student clicks witii the mouse on a 
particular object on the screen, a method is appropriately activated which may move the object, cause it to say 
sometiiing, etc. This is the basic flavor of all exploratory learning environments. It is the case, though, that we 
often want to introduce more teaching intervention into student actions. Control fiom the tutor is introduced 
into UngWorlds by the tutor object In LingWorlds the contio! is knit together by message-passing between 
objects and the tutor object A totally exploratory mkroworld has its control locally defined with the behavior 
of each object tied to student actions. A game microworld increases tutor control Finally, a goal-directed tutor 
microworW controls most of the interaction through a task-based agenda. Even in the goal-directed tutor, objects 
still retain individual control over the semantics of their own actions. For example, the tutor can be notified 
that a particular object has been dragged, and through additional message-passing widi the object determine the 
location. Thus, the bbjcct-orientcd paradigm allows an event-driven format that accommodates user-initiated 
actions as well as internally controlled actions. 

The LingWorlds Authoring System 

Briefly, the goals of Ae autiioring system were to enable a non-programming teacher of language to build 
microworM-bascd lessons. Thisisachievedby three types of novice programming: direct manipulation, menu- 
based selection mi a very easy to learn programming language. Using the autiK>ring system, functioning 
lessons can be devctopcd, tested, and refined very quickly. The authoring system allows immediate switching 
between cieating the mkroworld and testing it 

Direct manipulation is used to position objects in a scene and trace animation paths. Menu-based selection is 
used for choosing graphic images, selecting attributes of objects (such as whether they are draggable), and 
choosing system finctions such as testing a microworW. The programming language is composwl of English 
like actions and is «scd to create functionality for objects. Objects have a (growing) set of primitive actions, 
such ^ flash, tugkUght, say, and move. There are also control primitives such as (f and repeat while. From 



ihcsc primitive actions and controls, users can create new actions. Taken together, the built-in primitive and 
user-composed actions are available to all things and are called '^global actions.** Each thing also has a set of six 
"local actions,** which are functional responses of the specific thing to student events. The local actions are 1 
click action, 2 click action, dragging action, landing action, evaluation action, and Idstory action. The actions, 
both global and local, are written by the teacher in a scmantically based structure editor using hierarchical 
dialogues. There is no typing of text, simply selection of pop-up menu items. The system makes sure that the 
menu items presented to the user are syntactically and semantically appropriate. 

The authoring system produces sound by dynamically generating speech from a digitized phrasal lexicon. The 
sounds are recorded using SoundCap™ or SoundWave™. and then converted into resources which can be sela:tcd 
as arguments to the speak method. Thus, for example, one might use three separate sounds, **the lantern,** "is 
in,- and "the lifeboat,** which are suitably inflected to produce the unified coherent utterance 'The lanum is m 
the lifeboat** Sound generation can be context-dependent since an object can have a state history. Thus different 
utterances for an object can be generated in rcspo- ise to different situations. 

The LingWorids authoring system is an interpreter implemented on ^Ae Macintosh n computer in Allegro 
Common Lisp using Allegro's Object Lisp system. It generates Lisp code which can be supplemented by 
additional Lisp programming if so desired. TcchnicaUy, the system is buflt from a set of primitive features, 
which are either constructed in the authoring system (such as text) or are imported fi-om other programs (such as 
Soundwave™ and MacPaint™ ). These primitives include digitized sound* Macintosh-style images, locauons, 
integers, booleans, text, and icons. 

Evaluation u i 

Our informal evaluation of the system indicates that the average time to prototype a microworid is about several 
orders of magnitude less the time taken by Pascal programmers on the Mac (days rather than months). We have 
yet to extensively test the system with more formal evaluation studies, but those are planned for the future. 
Since die system is actually quite a powerful system for creating any instructional software, a version of die 
system was recendy requested and sent to Yale University for use in building instructional software for 
madiematics tutoring. 

We have built bodi an English and Japanese version of Provisioning the L^eboat, widi all die expected savings 
in programming. We built die exploratory version first, and then added the directed tutor version. A game 
version has not been programmed, but would be trivial. Since interactors and tutors can be specialized for any 
particular microworid. most of die code is inherited and reusable. The J^ese version of die Lifeboat }S 
virtually identical to die English except for die addition of animacy features required for speech syndiesis. 
Instantiation allows easy copies of objects widi die minimum of programrning effort We have also built a 
microworid called Flatland, in which the student is taught geometric shapes, colors, size and spatial relations. 
This is a directed tutor version, based on extensive protocols widi human tutors. The implementation of 
Flatland took one day, since almost all of the code was reusable from die Lifeboat problem. 

F. Summary and Conclusions 

In diis summary we have attempted to describe die principal outcomes and problems associated widi our project 
to build an interesting and effective computer-based second language tutor, LingWorids. Over die course of die 
project, we found diat die basic problems of interest to us in creating diis system increased in complexity and 
scope. However, die end result of dte project is die creation of a ICAI system diat teachers wUl be able to use 
coupled widi important new insights into the nature of language leanung and teaching. 
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Final Report 



A, Project Overview 

This FIPSE project was directed at improvements in undergraduate foreign language education at the Univerrity of 
Oregon (and similar institutions across the U.S.). More specifically, the project was directed at improving foreign 
language instruction at the bc^nning level and in the specific area of listening comprehension. At project 
conception, we believed that a new kind of computer-based support system could be created which would permit 
students to engage in listening oriented communicative interactions with simulations of real world problems. 

The principal outcomes of the project were: (1) software for use by teachers in designing and by students in 
interacting with language teaching simulations, and (2) new insight into the nature of language le£tming and 
teaching. Both of these have impact on foreign language instruction and on the design of other instructional 
software systems. 



B. Purpose 

Oral comr.iunication skills — the ability to comprehend and produce oral discourse — are crucial in nearly every 
educational, business, and scientific setting of language use. Yet the development of oral communication skills 
remains a difficult theoretical and practical problem, and traditional langur:ge teaching approadies regularly fail to 
help many learners. The central problem addressed by this project was to design a computer assisted language 
instruction system which could help beginning language learners develop their aural comprehension abilities. The 
reasons for targeting be^nners as well as listening comprehension were three. First, relatively little software is 
directed at true beginners. Second, relatively little software is directed at improving listening comprebensioiL Third, 
the computer environment is best suited to work in teaching listening. 

From the point of view of language teadhing theory our project draws on two important and iimovative approaches 
to second language lesiming and teaching: the communicative and the comprehension approaches. Proponents of the 
communicative approach argue that successfiil language learning occurs when the student is provided the opportunity 
to solve non-language problems using the developing second language (^^^ddov^rson, 1978; Krashen and Terrell, 
1983). They criticize traditional language teadiing for focusing too much effort on the conscious discussion and 
manipulation of rules of language usage and not enough effort on the acquisition of the second language grammar 
through efforts to use that grammar to solve actual communication problems. TWs philosophy integrates well with 
the general spirit of ICAI wherein learning is a problem-solving process. 

Proponents of the comprehension approach argue that second language learning is enhanced when beginning stages 
of language learning are devoted to developing the ability to understand the second languagp. Obligatory oral 
production is delayed until the student is able to understand easily utterances in the second language. Delaying 
production may even improve student performance in other aspects of ianguaae acquisition (Postovsky, 1977, 1979; 
Ashcr, 1966, 1969, 1977; Winitz. 1981; Winitz & Reeds, 1973). 

Our project embraces both of these complementary approaches to language learning and teaching. The instnictional 
system we have created, called Dngworids, involves the student in solving communicative problems interactively 
with the system. The student participates in problem-solving simulations which allow manipulation of objects in a 
physical scenario or rmcrowoiid Information about the problem to be solved as well as information about the 
microworid is &ven in the second language. Meta-level commentary by the tutor is also in the second language. The 
teaching intervention in these simulations can vary from highly directed to coaching to purely student-controlled 
exploration. 

Our pedago^cal model requires many small problem-solving enviromnents to be built, as well as many expert 
tutors. This motivated us to create a high-level authoring system. Our genera! philosophy in constructing this 
system is that we want the microworlds to be very knowledge-intensive and totally integrated with the intcrfaoe. We 
also want them to be reusable. These two themes have pushed us to envision a sort of library of microworids and 
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tutoring components. LingWorlds offers the teacher a rather large amount of programming power, if the teadier 
wants to use it, while permitting teachers with less experience the facility to build simple simulations. 

C, Background and Origins 



The historical origins for this FIPSE project involved the interaction and integration of ideas from four distinct 
disciplines: linguistics, applied linguistics, artificial intelligence, and computer science. Initially, graduate students 
in applied linguistics, in preparation to be English language instructors, were very much interested in the 
possibilities represented by computer assisted instruction (CAI or CALL, as it is referred to). However, the PFs, 
each in his^ier ovm domain, found existing i^ftware and the approaches to language teaching taken under them to be 
unsatisfactory, both pedagogically and compuUtionally. From a pedagogical viewpoint, existing CALL (computer 
assisted language learning) software did not incorporate innovations in second language teadiing approadies 
otherwise important to the field, in particular task-oriented, pioblem-solving approaches subsumed under the general 
communicative approach From a compuUtional viewpoint, existing CALL utilized very simplistic programming 
strate^es and control structures while not permitting easily truly creative teacher initiated variation in student 
lessons. Simply put, we found our students interested in the possibilities represented by CALL but we felt unable 
to recommend much of anything to them to study or emulate. 

Subsequentiy, we began to ulk about how an interesting CALL system might be made, and we played around with 
the idea of replicating aspects of Ashefs (1977) model of 'total physical response' training within a CALL 
environment A student of Tomlin's (Yuri Saul) had mocked up a very simplistic vocabulary learmng example on 
an Apple He which showed that the idea had possibilities. Review of this and further considerations by PI Douglas 
led to the belief Uiat an interesting CALL system could be developed using ideas from knowledge-based (artificia! 
intelligence) and microworids programming. 

The past 15 years have seen reasonable progress in delivering ICAI Gntelligent Computer-Assisted Instniction) 
systems which can be used in real pedago^cal situations (Anderson. 1985; Clancey, 1982; Sleeman, 1982; Soloway 
et al., 1981X ICAI systems can be contrasted with traditional CAI (Computer-Assisted Instruction) in the 
following ways: 

• The problem of learning and, consequentiy, teaching is seen as a cognitive, knowledge-intensive, and 

typically problem-solving process rather than a reinforcement process. 

• The control stnicture is dynamically generated by the interaction of curriculum, student response, and 

heuristics for diagnosis and tutoring rather than simply stored by the program. 

• The domain knowledge being taught is explicitly available for pedagogical decisions rather than 

embedded as numerical calculations (simulations or drill and practice) or blocks of tcxt/images/sound 
(programmed instruction). 

• Empirical, scientifically conducted studio of students and teachers form the foundation for the research, 

from initial data collection through to evaluation. 

Influenced heavily by the cognitive science movement, ICAI systems represent a significant attempt to model the 
cognitive aspects of teaching and learning, particulariy in providing an individualized approach. 

The notion of microworids was first proposed by Papert (1980) for LOGO programming. A later paper by Burton 
and Brown (1982) on reactive learning environments urged the pedagogical value of exploratory learning in domains 
otiier than programming. Our work has been greatly influenced by research on embedded semantics in microworids 
with direct manipulation interfaces: Programming by Rehearsal (Gould & Finzer, 1984), ARK (Smith, 1986), tiie 
latest woric on STEAMER (Hollan et al. 1986), and Thinglab. However, we intend to build microworids which are 
a simulation of Uie world as represented for tutoring purposes. This places our work closer to the work of 
Programming by Rehearsal tiian systems like ARK, STEAMER, and Thinglab which represent the knowledge of 
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Newtonian mechanics and hydraulics. We are interested in packing as much knowledge as possible into the 
microworld. Thus we want to allow the system to derive inferences beyond the facts explicitly declared. For 
example, we want the system to be able to compute spatial relations dynamically. We have found that AI work on 
scene analysb and diagram underitanding, as well as the literature on spatial reasoning and data bases is related to our 
work. Additionally, while we wish to continue the baac notion of exploratory learning, we also want to introduce 
more tutor-controlled strategies and ICAI computational mechanisms into the microworlds. 

For later purposes of generalizing microworlds as well as building the authoring system, we were committed to three 
design methodologies: rapid prototyping, taxonomic classification, and direct manipulation for the interfaces. We feel 
that these methods provide greater programming productivity through the ability to custom-tailor existing code by 
specializing subclasses and adding instances, and by immediate simulation of of code modules which shortens the 
generate-and-test loop. These three approadies are manifested in object-oriented programming? environments like 
Smalltalk and Lisp with Flavors. Previous work comparing rule-based with object-oriented ICAI tutors (Douglas, 
1986a) suggested that there are significant advantages possible with an object-oriented in^lementation, not the leabt 
of which is the accomnxxktion of event-ndriven, student-initiated control with event-driven, tutor-mitiated control 
Thus, our system, like Programming by Rehearsal, ARK, Thinglab, and STEAMER, is almost entirely object- 
oriented 

At this point, the ori^nal FIPSE pre-proposal was written, in some ways as a preliminary test of whether the ideas 
we were entertaining would be of interest to others. FIPSE seemed an interesting ftmding source because it was risk 
and innovation oriented and because it was pragmatically oriented. Both Douglas and Tomlin had substantial 
commitments to pragmatically oriented worL Tomlin was serving as Director of the University of Oregon's 
American English Institute (AEI), which provides English language training (ELT) to prospective and matriculated 
foreign students at the UO. Douglas had served as Director of an academic computing center. Both enterprises put 
us in touch with practical applications of researdi. It was at this stage that we began to reformulate our issues so 
that they would more direc^y serve educational purposes. 

Institutional context 

The narrow target of our project was imdergraduate students beginning foreign language study at the UO, a 
population of some 2100 students. These students were distributed in several languages departments: Romance, 
Germanic, Russian, East Asian (Japanese and Mandarin), and the AEI. We did not cultivate any preliminary 
relations with these departments (aside from the AEI), though we expected all to become interested as the projeci 
developed (which has in fact happened, except for the every small Germanic department). Thus, we knew who the 
targeted population of terminal users were, but more might have been done to engage language teaching faculty at an 
earlier stage. 

On the other hand, we did have subsUntial resources available to us through the American English Institute (AH), 
which PI Tomlin directed The AEI represents a rather imique context tor this FIPSE project Unlike most English 
language teaching units on major university campuses, the AEI, in its operational charter specifies that it must 
support and promulgate research in second language learning and teaching. In addirion, as a self-support unit in the 
UO College of Arts and Sciences, the AEI is able to direct material support to a project like this. Thus, in addition 
to the dnect grant support by FIPSE, the AEI provided substantial material support to the project in the form of new 
equipment and additional personnel support. 

Perhaps the greatest change in the context over time has been ancillary to the project itself but nonetheless important 
to its eventual successfol articulation at the UO. First, a change in Department head in the huge Romance languages 
department has resulted in a change in orientation regarding Romance language instruction. In particular, Romance 
has hired one new faculty member and is searching currently for a second specifically in the area of foreign language 
instruction. This has added some impetus to greater interaction between Romance and the AEI and Linguistics. 
Second, the UO has, through efforts by the AEI Director, secured a private ^ft of some $2(X),000 to be used 
specifically for the redesign and rebuilding of UO language lab facilities, including computer equipment. Both of 
these developments represent in^rovements in the UO context which will help set the stage for incorporation of our 
FIPSE project results into the UO foreign language instructional setting. 
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Project Description 

Overall, the principal goal of this project was to develop a second language teaching software system to assist nil 
proficiency learners develop beginning listening comprehension skills. We have built a knowledge-based tutoring 
system for teaching beginning oral communication skills for second (natural) languages. We entered the project with 
a number of elaborated examples of the kind of system we wanted to produce, but without great certainty about the 
precise path that would get us there. In some respects^ one of the most important outcomes of the project, besides 
the actual soHware developed, was the development of understanding of what some of the basic issues are in language 
learning and teaching thai must be faced in software development as v/cll as what some of the basic issues are in 
software design and implementation. As wc p^-oceeded, aKd even as we continue now, we redeSned our project goals 
to pursue some fundamental questions in this leaming^aching domain that conventional wisdom did not address. 

LingWorlds Ai a Tutoring System 

AN EXAMPLE: FROYISJONING THE UFEBOAT 

In order to provide the reader with a concrete idea of how the syilem works, the following is an extended example 
called Provisioning the Lifeboat In this simulation the student is faced with the non^linguistic task of provisioning 
a lifeboat before an ocean liner sinks. The simulation begins with an introductory animation and insl^iclions. The 
computer displays an animation showing an ocean liner approaching an iceberg. A simple oral narrative 
accompanies the animation. As the ship collides with the iceberg, a secorid, close-up animation occurs showing ^ihe 
sinking ship. Finally, an on -deck scene of equipment and people near a lifeboat is presented (Figure 1). At this 
point in the simulation, sti^dent interaction begins. Oral language directs the studtnt to locations of provisioas and 
equipment needed to use the lifeboat successfully. The student responds to the instructioas by pointuig. clicking or 




Figujrc! !. ProNisioning the lifeboat Scene 
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dragging with a mouse. Only through successful comprehension of the spoken language will the student be able to 
complete the tasks required to use the lifeboat 

While commands can be relatively simple to begin (Tut the anchor in the lifeboat."), they can become increasingly 
complex CTut the binoculars in the basket in the boat and then put the wateijug beside them."). Note that in this 
second problem the student must solve several complicated problems: know the names of the objects, perform the 
task in the correct temporal order, select which of the two baskets is the correct location, and understand prepositions 
of location (beside, in, etc.). 

Three vereions of Lifeboat axe described below which vary by the mix of student and tutor initiation and control 
complexity: an exploratory (student-initiated), a game, and a directed tutor. This demonstrates the kind of versatility 
that we want to provide for micioworld control. 

Explor^tarj LifeboMt In a totally exploratory mode, the student can use the mouse to single-click objects and 
hear their names CThe basket'') or double-click objects to hear a linguistic description of their locations in relation 
to another object (" The woman is not in the lifeboat'') The student is also free to drag objects from one location to 
another to hear the effect of a changed location. Help instmctions are available by clicking a question mark (?) icon. 
(See Figure 1.) 

Gftme Lifeboat During a game format, oral commands direct the student to locations of provisions and 
equipment needed to provision the lifeboat successfully. The sinking of the ship provides time pressure for 
completion of the tasks. The system keeps a score of successful tasks. 

Tutored UfeboMt The directed tutor n>ode supplements the oral commands of the game mode with remedial 
intervention. The control strategy that we use is derived from protocol studies of a human expert tutor. The tutor 
maintains a curriculum of concepts to be taught and a differential student model to determine state transition 
information and to diagnose types of errors. If the student fails to move any object after a few seconds, the command 
is repeated. A secoixi such failure causes repetition of the instructions for the overall task; after the third failure the 
system demonstrates the action. If the student moves an object to the wrong place, it is returned (by the system) to 
its initial location and the command is repeated. After several failures, the system wUl move the objea to the 
appropriate location, demonstrating the task- The student is then given another opportunity. 

These examples develop the comprehension-based grammar of the nil-proficiency learner in at least the following 
ways: 

• Developn^nt of lexicon^ocabulary. 

• Development of the grammar of spatial relations. 

• Development of the grammar of reference, 

• Development of basic syntax and word order. 

From the point of view of the student, we believe that the system we have developed will be challenging and 
intei^ting. The student cannot accomplish the needed tasks simply through knowledge of the worid, but m^jst 
comprehend the second language utterances. That is. the student is engaged in tme communicative behs-vior, using 
the developing second language to solve meaningful, non-language , .obleras-thc essence of the commuracative 
approach. The student demonstrates mastery of the linguistic task by physical manipulation of the world rather than 
by linguistic production. Thus, the student works on aural comprehension, the essence of the comprehension 
approach to second language learning. 
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E. Project Results 

This section descibes the LingWorlds simulation system, both in terms of its actual implementation and its 
assodated pedagogical insights. 

Pedagogical Uftorts 

Comolementan' to the general problem of developing the computer tutor are a number of pedagogical issues we 
SSXkg th^ pro&t. These efforts fall into five classes: (1) the nature of tutoring. (2) the nature of language 
SwSaS (3) specific curriculum and simulation lessons to be implemented when the system was ready, 
r4TgSKSdp£ fo! d^loping listening abilities at begimung levels, and (5) support efforts to the computer 
tutor direcUy. We wUl describe briefly the details of the efforts in each area. 

1 Tbc DMture of tutoring. Perhaps the most fundamental question for this project was posed by Do"glf early 

wLf fs uThaVtutors teach and how do thev decide what to do at any given point m a tutorial mteraclion? An 
SL"r ti ^ r^m^^^^^^^ question is needed if "the computer tutor is to perform in a way s milar to human tutors, 
^exi ting Uteratoe in S)mmunicative language teaching provides only the most general guidelines on ths 
guTdeS wWch are not specific enough to incorporate in any way in the tutor system. Consequently, we have 
devoted considerable effort to understanding the nature of individual tutoring. 

The communicative literature provides only the most general of assertions regarding what is tought and how that 
tea^STcSmplished. The communicative approach (VViddowson, 1978. 1979; Piepho 1981) view-s language 

a^Sve enterprise in which the learner enterUins multiple hypotl^ses regarding the structure and 
fimctioS of Urget language Constituents in natural discourse contexts until suffiaent contexuialized input is 
encountered to settle onand automate the learner's closest approximation of the native speaker norms. This process 
o?Swfconstruction of an interlanguage grammar (Selinker. 1972) is facilitated when linguistic in^ 
comprehensible to the learner (Krashen 1977. 1982). when it is of suffiaent quantity J^^J'^^^.s^n ^^ggj- 
contexts, and when the affective environment does not constrain exploration and nsk-takmg (Kiashen 1977, 1982, 
Schumann 1978). 

There are a number of 'tenets" of the communicative approach that can elaborate brielfy the general characterization 
pSJ^^ Under the communicative approach language is viewed as situated soaal activity as efforts of 
discourse production and comprehension, as comwunicatioii. Thus, in communicative language teaching: 

(1) Systematic attention is paid to functional as well as structural aspects of language (Utflewood 1981:1). 

(2) Classroom work is aimed at the simational and contex^feed use of language (Piepho 1981:20-21). 

(3) Teadiing and learning are made observable and transparent throu^ ^ 
through pictures, sketches, diagrams, and other representations (Piepho 1981:20-2 U. 

(4) Attention is focused on the ability to understand and convey information; i.e. on information transfer (Johnson 
1982:163-175). 

(5) The learner is seen a responsible partner in learning rather than as an object to be manipulated (Piepho 1981:11- 
12). 

Unguage teadilng represents the effort by the tutor to set up the conditions for learning described above That is. 
!SKre or le^fmely gramed teaching efforts, the tutor seeks to provide to the learner a 
comprehensible input drawn from a wide variety of genuine or nf^W^r^ 
Kralien & TerreU 1983) in an affectively 'supportive- enviromnent. While the observations above ^^^^^ 
of the general prindples defining the communicative approach, these general prmaples do httle to tell us exacUy 
what teadiers manipulate in tutoring and when and how they do it. 
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In our efforts, then, we developed an approach which yielded understanding of what it is tutors do when they teadi. 
Using a limited semantic dom^n, we videotaped expert language tutors engaged in one-oo-one tutorials with nil 
prdiciency leamere of various languages. These observations were taken both of face-to-face tutorials and of 
computer mediated tutorials (tutorials where, like the computer, the tutor did not have access to visual information 
on the movements of learner face and hands and eyes). From these observations, we have been able to extract 
significant components and principles of second language tutorials (Douglas, in press; Tomlin, Douglas et al., 
1989). 

(1) Tutorials are organized around a limited set of rhetorical acts. A rhetorical act represents a basic, composite 
unit of tutorial activity. The rhetorical act represents a basic linguistic action taken by an individual in a given 
discouree context. It represents an attempt by the speaker to direct some action on the part of the listener. A 
rhetorical act consists of three criterial components: 

• An intentiona] construct, or simply intention, 

• Behavioral content, 

• Preparatory conditions. 

The intention of a rhetorical act represents the underlying motive precipitating performance of the act. It is, 
following Brandt (1984), a menial event, the immediate and proxirrate cause of particular actions engaged in by the 
tutor. It is also the principal defining characteristic of particular rhetorical acts. 

The behavioral content represents the set of actions, mental or physical, which the tutor carries out due to the 
intention. The behavioral content of a rhetorical act includes a description of the generally desired outcome (goals) of 
the rhetorical act and of the means of achieving ti;is outcome (methods). For the rhetorical act DESCRIBE 
OBJECT, the behavioral content includes: 

• Goalj: Tutor directs learner attention to some part of the environment. 

• Methodj; Tutor points lo object. 

• Goal2: Learner links attended object to linguistic input. 

• Method2: Tutor utters object-name after attention is allocated to object by learner. 

Preparatory conditions represent tutor assumptions regarding preliminary or ongoing states of affairs in the 
tutor, in the learner, or in the world which must be present in order for a given rhetorical act to be executed. For the 
lingWorlds tutor, relevant preparatory conditions for many rhetorical acts include assumptions regarding: 

• state of the learner, 

• state of the curriculum, 

• comparison of (1) with (2), 

• state of the environnr.ent (world). 

For the act of DESC OBJ, the preparatory conditions include: 

• tutor assumes object is available to the learner, 

• tutor assumes learner is familiar with the concept of the object 

A rough model of a riietorical act, then, similar to the informal action model presented in Brandt (1984:128, is 
presented in Figure 1 Our descriptive efforts reveal that be^nning second language tutorials (within the domain we 
examined) are composed of a limited inventory of these kinds of acts. Thus, the building blocks of the computer 
tutor shoiJd be those rhetorical acts utilized in particular discourse domains. 
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Figure 2. Components of a rhetorical act 



(2) Tator eraJuMtioD Uctics. In order to decide what to do next, the tutor must evaluate current actions of the 
student. In our observations, the information used by ths tutor to do this varies according to the nature of the 
tutorial interaction. In both face-to-face and computer-n>ediated tutorials, it seems very dear that the tutor makes 
decisions about which act to select next prior to seeing the actual completion of a student action- That is, if the 
tutor in order to test some hypothesis like the student knows with certainty that 'atas means above' * asks the 
student to manipulate (in the target language Indonesian) a specific object with respect to another (Put the black 
square above the small while circle), the tutor will generally decide whether the student has gotten it prior to the 
action being completed. 

In both face-to-face and computer-mediated tutorials, the decision is made on the basis of the extent of uncertainty or 
hesitation in the student's response. The more hesitant the student is, the more likely the tutor will conclude the 
student does not yet have oxitrol of the relevant part of the grammar. However, the two conditions reveal difierenoes 
in the tactical information exploited to reach this decision. In the faoe-to-face tutorials, the decision is made 
tactically on the basis of information provided by a wide variety of responses: (1) latency in initiating response, (2) 
hesitation in movement of objects during response, (3) eye gaze, (4) head positions, (5) band positions. In the 
computer-mediated tutorials, the >dsual cues on the student arc not available, and the tutor relies primarily on latency 
and hesitancy in evaluating student performance. Unlike the assumption in virtually all CAl/CALL software we 
know of, the tutor does not rely solely or even primarily on the correctness of th<; student response. In the 
development of the computer tutor then, such observations must be taken into account 

(3) The tutor's model oftbe student Both the human tutor and the computer tutor must maintain a dynainic 
model of the student and what he has learned of the second language. In our studies, we embraced the general notion 
that students engage in "creatively constructing* the second language grammar. That is, on the basis of linguistic 
input which links utterances in the second language to observable situations in the worid, the student formulates (not 
consciously of course) hypotheses r^arding the grammar as used to effect communication- Additional input serves 
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to aid the student to reject false hypotheses and to increase the certainty with which he holds some hypotheses to be 
true. 

The tutor, then, must organize the tutorial effort to take this into account. Thus, the tutor must represent the 
student's current knowledge of the target language in terms of possible hypotheses and the strengths to which they 
are held And, the tutor's actions (selection and 'ose of rhetorical acts) must be tied to the formulation, rejection, and 
testing of hypotheses by the student. 

(4) Tbe role ofrepetitioB id iMoguMge lesming Mod teaching. There has been great interest in language 
learning and teaching in understanding the role of repetition (and more generally correction) in language teaching. 
Our descriptive studies have revealed a number of interesting hypotheses regarding the nature and use of repetition in 
language learning. First, v/e sec three distinct types of repetition: (1) repetition of the naost recent utterance (REP 
EXP), repetition of a selected part of an utterance QlEP FOC), and repetition cS an entire utterance/'action complex 
(REP ACT). While the three types of repetition share the gross structural property of amilarity in the utterance 
token, the three differ interestingly in their function and, hence, use by tutors. 

The first type of repetition, REP EXP, seems to occur without much, if any, behavioral input from the learner. 
Typically, then tutor produces a REP EXP immediately after DESC OBJ and independent of any request for the 
student to do something. For example, at the beginning of our Indonesian learning protocol, one can observe the 
following: 



(1) An example of REP EXP 



00:00 X on LBS 

00K)6 LBS 

00K)8 LBS lower right screen 

00:08 Ko:taky 

00:11 03 Ko:taky 

In this example, at the beginning of the tutorial (second OOKX)), the tutor produces the rhetorical act DESC OBJ, 
placing the cursor is on the large black square (LBS), monng the largp black square to the lower right hand corner 
of the display, and uttering an appropriate description of S:*? targeted object. Immediately afler the DESC OBJ is 
complete^ the tutor produces a REP EXP, repeating exacUy the previous utterance. There is no expectation on the 
part of the tutor at this point that the student will do anything, yet the repetition occurs nonetheless. It appears that 
the actual purpose of the repetition in this instance is to allow tibe student the benefit of retaining in workiing 
memory a good representation of the original token. That is, the REP EXP is a sodal act keyed into the difiiculty a 
bepnning learner has in sustaining an auditory presentation of novel linguistic input Such a hypothesis is 
supported by comments of learners after the protocols are collected which indicate how difficult subvocalizing the 
input was for them (even though they were able to perfonm the tasks required without great difRculty). REP EXPs 
are used also more at the beginnings of learning segments than in the middles or ends. 

Repetitions in which only a part of the previous utterance is repeated, what we call REP POC (repetition focus), 
occur at very different times for apparentiy very different reasons, REP FOCs occur after tbe tutor requests the 
student to manipulate some object and the student reveals latency or hesitation in responding. The RiEP FOC 
repeats just the taigeted lingiistic notion in that request Fro example, conader the data betow: 

(2) An example a REP POC (repeat focus) 

13:30 03 2.34 Ko:tak hi:lam ke.-cil (.36) DI-ATAS (.15) kotak hitam besar . 

/ 

13:35 05 1.11 DiATAS V 

13:36 SBS -> 

13:40 SBS a LBS 

13:40 05 3.81 Baik,/ 
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The utterance at 13:30 represents a request of the student by the tutor to put the small (kecil), black (hitam) square 
(kotak) above (di-atas) the laige (besar), bladt square. At this stage, the student has already mastered the lexical and 
structural aspects of Indonesian required to determine the referents of the noun phrases. It is predsely the the spatial 
relation of 'above* that is being manipulated in this instance. The student hesitates briefly after the end of the 
utterance (1.1 1 seconds after completion of 'besaf), and the tutor produces the REP FOC. Thus, the purpose of REP 
FOC seems to be to draw the student's attention to the linguistic item under review or examination at the preser; 
moment in the tutorial interaction. And, it seems that KEP FOCs occur as the tutor entertains a hypothesis that the 
student has failed to understand or to hear what had been said. That is, the tutor seems to believe that the student 
has made a mistake rather than an error. 

Finally, there are repetitions of entire acts, what we call REP ACT, which involves not only the repetition of the 
linguistic utterance associated with some manipulation of the simulated world but also requires repetition of the 
actions taken by the tutor linked to them. REP ACTs seem to be restricted in use to occasions where the tutor 
believes the sUident simply has failed to learn something, where he has made what is traditionally described as an 
error. Wtilt ordinarily errors are not directly addressed, but seem to be ignored in favor of new examples for 
consideration, tutors do occasionally repeat entire acts. We thus can see that REP ACTs are distinct from other 
forms of repetition in that they addtess error rather than mistakes or rather than assisting in sustaining a memory 
representation of the input, but we at present do not know when REP ACTs occur rather than simply using 
additional and new examples. 

These observations and others like them are described in a working paper, a part of which we will pi^esent at a major 
second language acquisition conference in February and hope to publish sometime after that. Most of these insights 
must still be built into the tutor sub-component of the overall system. 

2. Tbe development of model iMngmge teMcbiog MtmulMtions, IXiring the course of the project we 
developed a number of listening oriented language teaching simulations. They represent the kinds of simulations we 
believe will prove eflective pedagogically and enga^g to the language learner. An examination of these 
simulations also reveals something of the general nature of simulations for language teaching. Of these 
simulation" -nly the lifeboat has been implemented fully. 

(1) The lifeboat. In this simulation, which was discussed in the Project Description section, the student must 
provision a lifeboat in response to oral instructions in a second language (either ESL or Japanese). The student 
moves objects about the computer screen and places them in requested positions in the lifeboat. In order to do this 
successftil, the student must comprehend lexical expressioiB for the various items and relational expressions for the 
targeted locations. 

The lifeboat simulation has two component lessons. One is a traditional lesson in which the computer tutor 
controls the interaction, requesting acts of the student, monitoring student performance, and altering in interesting 
but limited ways the organization of the tutorial according to that performance. The second lesson departs fix?m 
traditional language teadung activities in interesting and important ways. In this second lesson, which we have 
called a prosponsivelesson, the student manipulates objects in the simulated microworld of the lifeboat scene and the 
computer tutor responds by describing the resulting state of affiiirs. 

(2) Maptiles. This simulation represents one of our ideal lesson environments. In tbe development of listening 
abilities it is quite commonly the practice to use maps of one kind or another to repre^nt a territory through which 
the student must navigate in response to input in the target language. Maps offer many advantages to the language 
teacher: (1) maps are good for information transfer and sharing; (2) students find map work engagng; (3) object^lace 
locations and ^Hrections given are concrete and observable; and (4) resulting procedural discourses are ridi as well as 
natural. 

The maptiles environment provides a toolbox with which teachers can construct their own maps and language 
lessons based on them. The principal component of the system is the MAPTILE, an object composed of three 
parts: (1) pathw^ays (which defme five particular maptile objects^ locations (in whidh building objects may be 
placed), and (3) building$ (which may have size, address, name, and other internal properties). The maptile inventory 
is illustrated below. 
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Figure 3. The inventory of maptile objects 



In designing a niap, the teacher builds a map by selecting maptiles and placing them on the display. Each maptile 
defines the pathways and other properties exhibited by each maptile selected Unspecified parameter are assigned 
default values. For instance, a given pathway might be designated as one-way, if the teacher so desired, or it could 
be lefl alone and assigned its default two-way flow of traftic Having defined a map, the teacher may also place 
building at locations on each tile, defiiung properties that each building may have (graphic appearance, name, 
address, hours, and so on). 

With a completed map, the teacher can also define a maptile problem. This represents the real world problem the 
student will be required to manage. Typically, such problems will be defined as NAVIGATION problems, in which 
the student must follow some specified path (as directed by oral input from the computer) under time pressure. The 
teacher defines the required path, the time limits or other limits on perfomianoe, and the language mput provided to 
the student For example, the teacher might CTeate a simulation in which the student controls a getaway vehicle. 
The vehicle must be maneuvered to a specific hideout along a particular path by following directions provide as oral 
input for the student. The student controls the cursor which may be represented graphically as a vehicle or pedestrian 
or other figure. 

(3) The animated dictionary. The animated dictionary (AD) is a lexical support system for students and 
teacheis. Students use the animated dictionary to search for lexi(^ items, related items and collocations, and 
variations from semantic prototypes. Teachers control the extent of student access in a ^vtn scenario. 

In the animated dictionary, lexical entries for ostensive vocabulary are represented by amniated sequences. For 
example, WALK would show an individual walking acrcss the screen and other manners of locomotion would be 
similarly displayed (kUN, SKIP, XX3. JUMP, TRIP, ctc.)» Students have two means of accessing the AD: (1) 
''meaning*' to sound (What is the word for tiiis?), or (2) sound to meaning (V\liat does tiiis mean?). 

Dictionary entries include a phonological representation consisting a digitized recordings of the item in a citation 
form as well as exemplars of its use in context. Th entries also include an orthographic representation, which may 
be represented alrng with or independently of the phonological representation or suppressed altogether. 

The graphic representation for each entry is composed of prototypical cases along with positive and negative 
exemplars. Prototypes present stereotypic examples of the requested categoiy. Pbsitive exemplars demonstrate 
typical and fringe cases. Negative exemplars demonstrate common misconceptions or confusions. In addition to 
tiiese cases, the AD also links a given entry to parallel and subordinate entries from the same general semantic class. 
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For instance, RUN is linked to parallel verbs of motion WALK, HOP, SKIP as well as lo subordinate semantic 
associates SPRINT, JOG, LOPE, eta 

The organization of the animated dictionary is a simple hierarchical one in whidi the student uses buttons on a 
Macintosh menu to select among alternative actions. The selections take the student to various other menus: 
BASIC ENTRY (a graphic, animated representation; a phonolo^cal representation; an oithogr^hic representation), 
VARIATIONS (positive and n^ative exemplarsX RELATED ITEMS (semantic associates and contextual uses), and. 
SOUND VARIAV?ONS (context sensitive variations in pronunciation). 

(4) FlatLand. In this simulation, the student builds simple t>\'o-dimensional configurations from a limited set of , 
objects: black or white, large or small, circles or squares. The tutor initially trains students to deal with the lexical 
items needed to ident'fjr mdi vidua! objects. It then introduces the specific spatial relations manipulated {above, 
below, lefK right, between). Fmally, the student is directed to build configurations of these objects like the one 
shown in Figure (4). This is done by placing one object at a time in its proper location in response to a single, 
complex utterance in the target language. 



(5) Mystery world. This simulation places the student in the position of a witness to some set of events. The 
events are presented as an animated film of some kind or other. Associated with the film is a descriptive sound track 
which relates the actions witnessed by the student This narration, whose contents are provided by the teacher, can 
precede, follow, or occur simultaneously with the movie. 

The student engages in two kinds of tasks. First, the student practices ordinary listening, matching the narration 
with the ongoing events portrayed in the movie. Second, and more interesting, the student can then interact as 
wimess lo the computer tutor's detective to solve a mystery presented in the movie. The detective can interview 
other witnesses in the scene and the student must listen and verify the truth and accuracy of those interview 
responses. In addition, the detective can interview the student directly, through the judicious use of yes-no questions. 
This interaction can take the student through the traditional hierarchy of question types in listening: (1) questions of 
observed fact, (2) questions of inferred fact, (3) questions of inferred motivations, and (4) questions of evahiation. 





Figure 4. A fmal FlatLand configuration 
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This particular simulation has been mocked up using Apple animation for one scenario. In this scenario the 
studenlAvitness observes several people entering and exiting a bookstore. A robbery occurs in the bodtstore (unseen 
directly by ti« vdtness), but the timing of events pennits the witness to infer who the culprit was, 

(6) Minor simnlatioiis. In addition to the major simulation types described above we also considered and in one 
case partially implemented several other more minor simulation types. One example is a simulation of the face. By 
pointing to parts on a human face, the student is able to hear the lexical names for the constituent parts of the face 
(eyes, nose, mouth .etc). This has been implemented. 

Other simulations examined included: 

(1) ClothesWorid: in which the student travels through a mall to acquire climate appropriate clothing under 
a limited spending budget 

(2) Plumbing: in which the student builds a water s>Gtem out of plumbing components in response to 
instructions from the tutor. 

(3) ForestWorid: in which the student manages forest resources according to orally presented information 
on the stale of the forest and possible outcomes of alternative dedsions that might be taken at a given 
moment 

(4) BusinessWorld: in which the student manages a simple manufacturing scenario, buying raw materials, 
producing manufactured goods, and selling these goods in a competitive market. 

It is our belief that these curriculum efforts represent important ways in which the basic computer tutor system 
might be used effectively. As the project continues, it is our hope that the first four simulations types above will be 
implemented, both for their own merit and as examples for others to modify and worft: with. 

3. Tbe aature or simulitioas. In order to assist teacher in the design of engaging and effective language 
leadiing simulations, we examined the underiying organization of task-oriented language teaching simulations and 
developed a taxonomy of simulation problem types. These include: 

(1) Resource Management Problem*: the Student manages a finite quantity of resources, seeking to maximize 
their utility under varying constraints and conditions of use. 

Examples 

Budget management ( In order to accomplish some GOAL, the student expends portions of a finite MASS 
VARIABLE (money, time, energy)... 

Inventory management ( In order to accomplish some GOAL, th? student distributes individual items from a 
limited INVENTORY of ITEM TYPES (auto parts, flora, f una, etc.)... 

(2) NaTigation Problenu: the student maneuvers through some medium or territory as directed by the tutor, 
reaching specified goals and avoiding specified obstacles. 

ExsuTjples: 

MAPTILES: described above. 

AdTcnture simuIatioAs: the student travels through a PHYSICAL WORLD (dungeon, forest, shopping mall, 
tourist zone, Paris) collecting ARTIFACTS (gold, goblets, butterflies, bric-a-brac, souvenirs, insults) 
depending on the successful interpretation of INSTRUCTIONS (warnings, suggestions, reconunendations, 
etc.) provided by a TOURGUIDE (imp. ranger, store proprietor, taxi driver). 

(3) Racc/choc Problems: these represent collocations of linuted resource problems (in particular TIME) with 
navigation problems. 
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4. GcncTMl principles of communicative IsnguMge teaching Urgeting listening comprehension. 

We effected a scardi of the pedagogical and theoretical literature in second language learning and leachbg and we 
interviewed practidng second language teachers in order to create an inventory of listening activities. The inventory 
of activities is organized to reveal: (1) the necessary prerequisite skills required to perform the task, (2) the learning 
goals for the task, (3) the expected outcomes for the task, (4) the means/imetbods of prcsenting^exccuting the task, 
and (5) means of evaluating the effectiveoess of the task. WMe no origmal work was done in this area, it remains 
useful in organizing suggestions for practicing teachers who might be interested in using the computer tutor system. 
These insights will be reflected in the fmal LingWorids manual. 

The LingWorids SimuUtioo System: Implementation 

The LingWorids tutoring system consists of two component parts. . One component is seen and used by the learner 
to engage in language learning simulations. This component we will call the tatorial system. The second 
component is used by language teachers to create simulations and language learning problems. We will call this 
component the authoring system. 

The Tutoria! System 

The LingWorids system is illustrated by functional parts in Figure 5, The microworld component is essentially all 
object-oriented, as is the tutor component. The animator and language generator contain special routines which 
manipulate digitized data and are written in assembly language to be as eOlcient as possible. The language generator 
has a digitized lexicon, and a case-frame semantic grammar, and is capable of dynamically generating utterances from 
a conceptual representation. It much too primitive to be considered a full-fledged natural language generator but is 
sufficient for our simple language teaching. A separate generator is needed for each language taught. The animator is 
a specialized unit for nmning long animation sequences as "movies" rather than generating them directly in the 
microworld. However, the microworld objects are capable of simple animation sequences. 



uWorld 



▼ ^ 

Animator 



INTERFACE 

Figure 5. FuncUonal Description of UNGWORLDS 




Language Generator 
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Contexts and Scenes: Each lesson or exercise consists of a sequence of contexts. Each context is either a 
"moNie** sequence or else a "scene**. Within a context, a series of movies or scenes may occur. Each scene can be 
composed of subscenes appearing simultaneously in the same window or in recursive windows. A scene is simply a 
set of problem-solving objects for the student to manipulate and to focus attention on, A scene may have a 
background whidi is simply a bit-mapped picture and not a full-fledged object 

Inteimctors: Each object rc|Htsentcd in a scene is a highly specialized object called an "intcractor'* olgcct 
Interactors in the lifeioatnricroworld (Figure 1) include the rope, the anchor, the lifeboat, and the question mark (?) 
icon. Interactor instances have various features, actions and relations with other interactor instances. Interactors are 
usually visible as a bit-mapped picture and arranged in x-y locations during the initialization of a scene. They can 
also beconae invisible. An invisible interactor can mark a spatial region to which other interactors can be moved. 
Interactorr. are located on planes with each interactor occupying its own plane. Interactors can swap planes 
dynamically. This provides the scene with a simulation of three-dimensional reality as interactors are animated by the 
system or dragged by the user. 

Interactor Actions: Interactors can respond to both user actions, such as mouse clicks, and system actions such 
as message passing. Interactors can highlight themselves and they can speak. System animation for the interactors 
can result from following any of three methods: an arbitrary path stored by the designer, a computed trajectory, or a 
location described by a natural language spatial relation, which allows a description of motion by displacement It 
should be en^hasized that these interactors, while representing concrete entities in the real world, do not manifest the 
physical laws of that worid Interactors do not "^fair under the influence of gravity, unless caused by the system 
designer. In other words, the microworld is more of an imaginary, linguistic worid Each interactor responds to 1- 
click, 2-dicks, press-and-hold, and press-and-drag mouse actions by the user. The designer can choose to enable or 
disable these methods. Thus the system can enable drag actions on some interactors and ignore them on others. 

Semantic Properties: Each interactor has a list of semantic feamres that define it for linguistic purposes. These 
features include whether or not the interactor is animate, a person, an artifact, a vehide, a vehicle of public 
transportation, or a container. Any other attribute-value pairs can be added by the designer. For example, an author 
could specify the interactor's color or the lexical item that designates its name. In a Japanese version of Ufeboat, 
animacy features were added to interactors for purposes of generating the correct morphemes during speech 
generation. Eadi interactor can also be decomposed into a subset of other interactors. This expresses the has-parts 
relation. For example a human interactor can be decomposed into further interactors of arms, legs, torso, and head. 
This relation can also be used for generalized possession as well. Interactors can also have ports which cause auto- 
message passing betweeu them when they are contiguous. 

Spatial Relations: Probably the most difficult and interesting part of the system is the set of spatial relations 
that we have very thoroughly studied and built into the system, Altliough this was done initially for the pedagogical 
goal of teaching spatial relations involving prepositions, it has had interesting side effects for the overall design of 
the system. For example, it allows interactors to locate themselves in a scene relative to a spatial relationship with 
another interactors. 

Our initial work with spatial prepositions included those in the essentially horizontal and vertical two-dimensional 
planes: left cf, right of, abovG, below, and bem^een (Because of issues hinted at previously, our simulations are 
primarily two dimensional but because of the possibility of overiapping and movement, they approximate three 
dimensions. At this time we have yet to implement the depth plane prepositions: in front of and in back of) 

A major problem for us was where to put the deictic ori^n of the speaker. Is the speaker, in this case the tutor, 
describing the scene from the position of the student looking at the scene on the CRT screen, or is the speaker 
sitting opposite the student? An argument can be made for either case. Certainly the voice is coming from an entity 
seated across from and facing the student. On the other hand people using deictic expressions can always see the 
location of the speaker and make the relative orientation adjustments between what the speaker sees aixl what the 
hearer sees. In the case of the computer, one is simply not sure where the speaker is. The least ambiguous 
assumption is that the descriptions are from the point of view of the speaker in a position of the viewer since the 
location of the speaker isn't known. Thus spatial relations are computed for a scene from the deictic perspective of 
the student 
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dJiSSin? ^ ? ' °^ ^ it is the case that judgements of these relations can vary 

aependuig on shape, size, discourse task, and contextual arrangement. There also may be prototroe oositions for 

S r«TaS Jr T^^" r " 'S"^' °' ^""^ usfnglrpS^m'^Sc shaSs to 

teadi the relations of kfl. nght, above, below, and between (Jom^m, Douglas, cL al., 1988). Based on these 

protocols, our own intuiUons and psycho-physical experiments, we have developed algorithms to comoute these 
and tutoring strategies have been moorporated mto a mlcroworld simulation caUed Hatfand 

In order to implement spatial relations in our tutor, we compute on demand for each interactor several soatial 
prowrties: center-of-area (centroid), distance, areas that project from edges, and angular displacements. These 

^ ™P f"«"^ soine ^iher straightforward message-passing algorithms, have sufficed to compute 
succcssMlythc relations M right, above, below, and a modified two-dimensional in In some 1 rSiSi^S 
proxmiitj' becomes a crucial factor for the computation of these relations and appear as thfmost JSt 
component to determme in the general case, since it is clearly dependent on social and psychological factors We 

«s;'fX"f?;n-^^^^^ " '""^ ^"'"^^ ™^ ^p«^««^-n °f ^ 

Si!^ °f ^^T^ ^ generalized tutors for various languages, it seems reasonable to allow the 

teachei/author to change the semantics of spatial relations as they may vaiy by languages. 'WTiile this might seem 
tevial to mono-hngual English readers of this paper, it be a Wand difficd^blem Fo5examT^^^^^ 
a bowl which contams an apple. Both English and Chinese speakers would say -TTie apple is [n Ae towl " Now 
STh : J' """^ "S^ ^"P^'^^y '""Sined the first one in ifcanorLTonViTutior) 
Sw fhnt ?iri ^'f/P-^ "^l' ^'^^"^ ^ '^^^ ^-i^W" the containment of 

i^taSiL ^i^^Tth^t « "^11^^ "P?'' C^^^^y P^'blems occur in 

S^Jint ??i- "^^"'^ ^^"^^ 'P""*^"" «^ someone top of the bus. 

S,^p1.^? differences suggest that the knowledge representations for spatial relations will have tS be open to 
change by the authonng system. This could be an exceedingly difficult goal to achieve. 

Integrating the Microworld with the Tutor 

S P^jf^j"^ description or the microworid components describes how program code is modularized by what is 
^mially display and control of the interface. Since much of this is generalizable, it is highly producUve to have 
Sn?" ? IT Wi d^f. ^finitions. However, as just descri^. the basic control lS>ears tXprimS 
ft^ent initiated If Uie student clicks with the mouse on a particular inteiactor on the screen, "method is 
appropriately activated which may move the interactor, cause it to say something, etc. This is the basic flavor of all 

Lto sSSt SXTaT- " ''''' ^'^ teSg i^te" :itton 

mto student actions. TTius, ICAI systems vary along a continuum from totally evem-driven exploratory 

""t 'i^'* ""'^"^ ^l"^" ^ ^"^^ t° goal-dire«ed tutors which have a highly 

sp«ified contra stmctoe. Teaching expertise includes how to tutor, what instnictional approadi to use, and why and 
^ften to tutor the student. Insights into the complexity of language tutoring are dSbed beloTas weJ al L 
Douglas (in press) and TonJm. Douglas, et al. (1989). 

Control from the tutor is introduced into LingWorids by the tutor object. In LingWorids the control is knit together 
S^LT 5?Ti" '"^"f i^jf^ts and the tutor object A totally elplor^toiy micraSi lS SntSl 

Sil fri^nSi^?.^^'''"' "^"^t '° ''^^^"^ A game ^ucroworld increases tutor controT 

Finally, a goal-directed tutor microworid controls most of the interacUon through a task-based agenda. Even in the 

V^ZTT'ST"" control over the semantics of their own acSrT^r exaMe. 

the tutor can be notified that a particular interactor has been dragged, and through additional message-passSg S the 
interactor determine the lo^tioa Thus, the object^riented paradigm allows anlvent^riven f^S 
accommodates user-mitiated actions as well as internally comrolled actions. 
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Tlie Authoring System 
Cluracterittics of Authoring Syitems 

SfwS'S Jf/tSThtiff •T^'''''^^' ^*^°°^y. a tiine-consunuDg and intensive 

taskwhich has al of the problems of interface construction. With the advent of bit-mapped displays and mice the 
author ^ faced with ever increasing design complexity as graphics and sound supplement SS wiiSs 

totolisp on the Xerox llOO's, the mterface programming effort has been informally estimated to consume about 

^xTfP^f ."^"^ ^ Macmtosh can similarly be attributed to the complexity of composing oveTfiOO 
ROM-based interface fiincUons into usable interactive programs. composing over ouu 

The reasons for this increased complexity are due to several factors. First of all, designing an interactive interface is 
not an exact engineering task, and much less a science. Although the demandb foSSSS^mpuS 
SS^K*^-"" of experimentalpTychology liteiature, it stUl ^mafnsX^owKe of 

^ " P'"^ Consequently, ^e author must still rety uW^ gSS^ 

tet methodoloffiT that introduces users into the design loop early in the process. A «cond Son^ S^SS 

SXZ^^ fe^' algorithms, perhaps even animation. These techniques a« not well-faiown byTa^erage 
programmer and need constam visual verification during tiie programming process. Thinfly the codiii effort for 

S^rmaltt^'^-f -P^^^^^ user-SefinaL^windo."! I enormo^^! tiiS nSjy^t^ have 

wmdow managers to assist m the mn-time association of mouse and keyboaixi events to particular windows and 
menus the author may still have to handle much of tiie control in tiie application code.Sy ^^^ts of 
^^'^ are dynami^y generated by the user ^nm-time contribute toVevSSven 
program tiiat is not easily represented m standard procedural languages such as PASCAL. In classical oroceduml 
SSS' 7T ""^ P"'"""'^ f"''" The state^l program determine ataoSel^sSte of 

LT^r., ^'ZT"^"^ !L?"''^ ''^^ °f *^ '^''^^'^ ^ '^^U ^ the state of the pS^m beX 

SprSrtolJl.TtSr'T'^'r than data-driven programming (Shaw, 1986). EvL Sidard 
X^lSiSSl^t 'I "^i ^^-T"^^"" with state transition diagrams (cf. Jacob, 1985) lose their usefulness 
as the sequential natoe of text-style dialogue interaction disappears to be replaced by direct manipulation. 

U^°iS,£rii^ * of research on reducing tiie complexity of interface programming by developing 

iTSSTc^ti^fn nf '^•^'"^.^'''^ "^'^^""^^P ^^-^^ funSonality of the' wSS^n and 

What f«*^'«^ have bera developed for interface design, such as Trillium (Henderson, 1986), AIDE (Hix & 
nf'Si^f^ and P^ER (Helfman, 1987), ^nd to have limitations caised by spe^ializii^^^a piS^r type 
of mteractive fonnaL For example, Trillium is used to design copiere and AIDE. daloWiSction ThT^ 
hmitations make it difficult to generalize these systems to the cation of microworids fo. lingSc inte^on. 

DKign is a higUyknowledge-intensive activity which results m the creative production of an object or process. 
^;nti?^,^H.n emphasize top-down refmcment bcgimimg witii a specification and proceeding to tiie 

1? J^' P'^Pt've approach ignores many of tiie bottom-up practi^of working designers. For 
example human designers are more opportunistic and domain knowledge intensive, such as using an existim? desian 

K^^LtfoSS' i^'^^r^^'^--' ^^'^^ study of software' designers hSSbSSi^ 
Sfnu^. of simulating tiie specification as an attempt to ferret out possible specification bugs. FinaUv, 
m fcwwhT^,^"^ °" specification. Thus, the prevailing utihzation in practice (as opposed 

to theory) of what are called rapid prototyping methods for design. Therefore, it is important to let the amhor ofT 
anguage le««>n quickly build and tiien refine tiie microworld in whidi tiie tutoring intSaction wS Uke p a«. T^ls 
for the creation of graphics and sound already exist; what is lacking is an environment which easily provfdi 
programmatic firactionality for these elements. ^ provioes 

K^f ffi?^' 5!?!^^?' ^Adelson & Soloway (1984). Kant (1985), Kam & Newell (1982), and Steier & 
lUnt (1983) m the domain of software engineering is the following: 
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(1) Dedgneis rapidly develop a kernel idea and refine it during the design process. 

(2) Deagncrs spend about half of their Ume simulating the behavior of their programs. The simulation process 
serves niany functions: it helps the designer integrate oonsUtuents from several parts of the design; it seives as 
a kind of agenda to keep tiack of subtasks that require attention; it encourages a kind of balanced, methodological 
refinement of the software system; and it allows for comparison to the design goal. Simulation helps the 
designer identify interesting opportunities for improving the design. 

(3) Designers take both mental and written notes on things to remember later in the design such as constraims, 
partial solutions, and potential inconsistencies. These were not handled immediately since they v;ei^ at a greater 
level of detail than the current state of the desiga In practice, the designer is frequeiiUy able to avoid problem 
solving the entire design by recalling previous partial solutions 

(4) . This suggests that a support system for design should provide examples of common parts and previous designs 

which can be canmualized. This supports the idea of inheritanos through class specialization or prototype- 
modify found in object-oriented languages. 

It has also been observed that designers make mistakes which they do not recognize until far along in the actual 
implementation process when change may have a radical effect on previous design choices. This suggests that 
simulation of the design as eariy as possible is crucial for debug^ng it . In a nutshell^ this is the justification for 
techniques of rapid prototyping. For mechanical en^neers drawings play an important role in the process of design . 
They act as completeness checks, simulation, and analysis. Interface designers no doubt gain the same value by 
direct graphic layout of the interface objects. This supports the notion of a direct mariipulation approach to design. 
The objects which are to be Uie context and referents of the linguistic interaction should, then, be manipuiable by the 
lesson author in much the same way that objects can be manipulated in the microworld itself 

We believe that much of the effort and complexity involved in lesson design and implementation are be reduced by 
the introduction of rapid prototyping using object-oriented programming. In addition, we believe that a system for 
design of the lesson interface should provide a specialized environment for this iLspect of programming. 
Furthermore, we believe that this UIDS should incorporate, wherever possible, direct manipulation techniques as the 
fundamental presentation to the author. The system that we have developed thus incorporates two programming 
methodologies, direct manipulation and object-oriented programming. 



Object-oriented Programming 

The message-passing nature of object-oriented programming languages, exemplified by languages such as Smalltalk, 
Lisp with Flavors, and NEON, an object-oriented Forth, allow easier representation of the event-driven nature of the 
modem interface. Class inheritance increases programming productivity by allowing design by taxonomic 
classification and specialization, with less duplication of common code. Multiple copies of a defined object can be 
easily and dynamically instantiated to further reduce programming code. Thus, development time can be reduced by a 
factor of four or five (Schmucker, 1986). Lesson authoring becomes an effort of selecting from a library of reusable 
programming parts. This encourages consistency across lesson design. The encapsulation of functions within 
objects as cleanly interfaceable modules reduces progranwning bugs. 

Although all object-oriented systems offer an extensive collection of interface "pieces** such as a window package 
that tiie programmer can use to build the interface, only Smalltalk has a well-^veloped and systematic approach to 
die user mterface incorporated into a system called the Model-View-Controller (MVC). This system provides the 
programmer with ready-made text editor, scrolled windows, pop-up menus and process scheduler. Unfortunately, the 
MVC is rot documented in any of the thrte currently available Smalltalk books, a fact which makes it unavailable 
even to Smalltalk programmers.\\'hile the Model-View-Controller holds promise as a viable system for interface 
design, our laboratory experience with it has shown it to be less than desirable. The separation between these three 
elements is not often easily made, i.e. controllers have views, and view information is frequentiy still embedded in 
the model (the application code). A spatial mapping between the displayed object (view) at which the user may 
point and the model object does not exist. The lack of an extensive collection of interface flmctions such as dragging 
and simple animation. Finally, all since views are dependent upon receiving change messages from a model, an 
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mefTicient synchronous system of sending messages to all views is required. While several object-oriented systems 
such as STEAMER (Hollan, 1986), ThingLab, awi ARK (Smith, 1986) provide some unusual approaches to 
nKxlifying the user mterface, none are as extensive as a Smalltalk system called Programming by Rehearsal (Gould 
& Fmzer, 1984). It was developed for non-programming teachers to build simple educational games. Programming 
by Rehearsal does not use the Model-View-Controller system of Smalltalk, but substitutes its own mterface model 
whose metaphor is that of a theatrical production. Using a primarily visual programming enviromnent "perfomiers" 
(objects) can be moved around on 'Stages'* (windows) and taught how to interact with cadi other by sending "cues** 
(messages). Performeis have simple animation and picture display as part of their definition. 

Class inheritance Is finessed by using the concept of prototype. Thus, the basic definition is to copy an existing 
object aixi then modify it, rather than define a sub-class and dedare an instance. 



Direct If anipulation 

Object-oriented languages usually eiK»urage a rapid-prototyping style of program development that allows 
incremental visual verification of program modules by the programmer. The programming environment is richly 
endowed with integrated cditois, browsers and debuggers. However, interface design is done througli traditional 
textual programming, not by a form of direct manipulation. Direct manipulation environments provide the user 
with several highly useful features when performing a complex task: continuous representation of the object of 
interest, physical actions or labeled button presses instead of complex syntax, and n^id mcremental reversible 
operations whose impact on the object of interest is immediately visible (Shneiderman, 1982). 

The incorporation of direct manipulation interfaces into authoring systems (and programming generally) has taken 
two forms: visual programming and programming by demonstration. The former emphasizes the visual 
representation of the code (Myers, 1987). In describing an interface spcdfication system, Jacob (1985) recommends 
tl^ use of graphically displayed state transition networks as deagn notation over BNF grammars because of user 
interface ease, emphasizing the importance of visible graphics to represent abstract entities over linguistic 
description. Programming by demonstration, emphasizes the physical actions of the programmer to capture the 
actions of the user at the interface (Myers, 1987; Gould & Finzer, 1984). 

A direct manipulation design environment for programming the interface allows the designer to specify the interface 
both visually and by manipulation in order to automatically generate the code. For example, a window can be built 
out of window parts with sizing done by manipulation rather than textual spedfication. For design in areas where 
spedfication languages may be difficult to implement (for example, graphic design) or where it is difficult to 
formally spedfy the effident implementations (for example, human computer interaction), direct manipulation rapid 
prototyping systems appear preferable to textual parameters, A spedficp.Uon of the user's possible physical 
interaction can be done by emulating that interaction (Myers, 1987). 



How The LingWorlds Authoring System Works 

Following the precepts delineated in the preceding section, we developed the LingWorlds Authoring System. 
Briefly, the goals of the authoring system were to enable a non-programming teacher of language to build 
microworld-based lessons. The system integrates images, sounds, and interface functionality in an authoring system 
for second-language lessons. The authoring system's prindpal characteristics include object-oriented prototyping, 
making interface-related functionality available to all objects, and providing a language for object functions powerful 
enough to encompass domain functions in the interface. Using the authoring system, functioning lessons can be 
developed, tested, and refined 

System Overview 

The LLngWorids authoring system is an interpreter implemented on the Madntosh II computer in Allegro Common 
Lisp using Allegro's Object Lisp system. It generates Lisp code which can be supplemented by additional 
programming. The system is built from a set of primitive features, which are either constructed in the authoring 
system (such as text) or are imported from other programs (such as graphic images). These primitives include 
digitized sound, Madntosh-style images, locations, integers, booleans, text, and icons. 
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•Tutor" menu pemiits the user to create and develop the tutor in the Aether window. Finally, the "Windows" menu 
lets the user toggle between the World and Aether windows. Many of the menu choices are shadowed by 
mnemonic Macintosh-style command-key combinations. 
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Figure 7. Menu Selections for Things 

LingWorld's things are prototypical rather than classed That is, every thing is created with all the characteristics of 
the basic prototypical thing. There is no notion of mheritance or class-based specialization- The various aspects of 
things arc determined by a user through direct manipulation, menu-based construction, or in an English-like language 
using structured dialogues. 

Direct Manipulation 

Direct manipulation is used to determine positioning and paths of things. The user can simply drag a thing to a 
desired location in the window. Things can be displayed in different forms, including image, icon and text Things 
not only have the static notion of location but a related dynamic notion of path as well. Each thing can 
automatically record and then follow a sequence of poations. Such paths can either be recorded demonstratively by 
dragging the thing with the mouse, or by specifying exact locations. In both cases, the path can be edited using a 
suitable dialogue. Figure 8 shows the author editing a thing's path. Multiple tilings can follow tiicir paths 
simultaneously to produce simple animations. 
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Figure 8. Editing a Thing's Movement Path 

Menu-Based Conitniction 

Menu-based construction is used for specifying the thing's graphic representation. Each thing can have a graphic 
image, a textual representation, and an iconic representation, although only one of these can be displayed at a time, 
of course. Thing n^nu commands allow the user to choose an image and a mask for the graphic image 
representation, to set the text for the textual representation, to select an icon for the iconic representation, or to 
choose the displayed representation of the thing from any of these. Menu-based construction is also used in building 
the lexicon, creating and deleting things, and setting attributes of things. 

Each thing is created with a set of seven built-in attributes. Attributes are analogous to instance variables in regular 
Lisp things, but are strongly typed Each attribute has an initial and a current value, and is defaulted appropriately. 
The built-in attributes are position, which is the location of the thing in the Interface window; highL'ghted?, a 
boolean value indicating if the thing's representation is in reverse video; visible?, a boolean value indicating if the 
thing's representation is visible or invisible (invisible things are useful for things like "hot spots" in larger things); 
draggable?, a boolean value indicating if the thing can be dragged when the interface is in running mode; dick i?and 
click 2?y boolean values indicating if the thing will execute its response to being clicked or double-clicked in running 
mode; and plane, an non-negative int^er indicating the foregiound^ackground position of the thing's representation 
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relative to other things in the window. New attributes may also be created by the user, who must specify the 
appropriate type. 

Adding Functionality to Things 
Related Languages 

In developing the action language for LlngWorlds, we looked at a number of user interface design languages, 
including Experlnterface Builder (for ExpeiCommonLisp), a system developed by Luca Cardelli (Cardelli, 1987), 
HyperCard, and Programming by Rehearsal (Gould and Fmzer, 1984). 

Experlnterface Builder is built on top of Macintosh ExpeiCommonLisp and was developed by Jean-Marie Hullot. It 
allows the designer to select interface objects (buttons, text box, scroll bar, etc.) from a palette similar to MacPaint 
and drag them to the desired location in a window. Each object is associated with a Lisp function that v^ill be 
evaluated when a user event occurs. Graphic images can be imported from MacPaint to ^ve individualized 
appearance to the interface objects. Testing and debugging can be rapidly done since the interface design is entirely 
within the ExpeiCommonLisp environment. 

The system developed by Luca Cardelli is similar to Experlnterface Builder but allows a more general concept of user 
interface objects called wteractors which are located within a higher level object called a dialog. Each interactor's 
appearance is cuslomizeable by location, size and graphics. Inieractors can be composed into groups. Interactors 
communicate with the application by way of an abstraction of user information called the ewnts. Events are defined 
as having attributes, state, and status. This allows the application to avoid low -level attention to mouse and 
keyboard actions. The interface designer has available a cfialog editor which allows mouse-based selection of 
interaclor instances and customizable appearance. Cardelli's system was built for use with Modula-2+ and runs on 
the Firefly personal workstation. 

HyperCard is a system recently developed and released by Apple Computer for the Macintosh. It is intended as a 
programming system for novices. The system is object-oriented and based on ObjectPascal. What is exciting about 
it is that it provides a direct manipulation interface design system which is very easy to use. There are six major 
kinds of objects: staci, card, button, field, background and picture. The first five are fiist-class objects, in thai lhc> 
can have scripts containing message-handlers. Picture is a second-class object which can display an image but does 
not allow user interaction. (Note that this diffets from Experlnterface Builder.) Cards are objects that are linked to 
form a tangled hierarchy. They are essentially a window and can contain the other types of objects. The script 
writing language, Hypertalk, is a combination of Pascal and Smalltalk and has about 40 basic commands and a 
reasonable set of control structures. 

Action Language in LingWorlds 

The LingWorlds authoring system's English-like language for methods is used to create running-mode fimctionality 
for things. Things have a (^wing) set of primitive actions, such as flash highlight, say, and move. T>.ere are 
also control primitives such as if and repeat while. From these primitive actions, users can also comprse additional 
actions. Taken together, the built-in primitive and user-composed actions are available to all things J»:id are called 
•'global actions.** Each thing also has a set of six "local actions,** which are functional responses of the specific 
thing to run-time events. The local actions are / click action, 2 dick action, dragging action, landing action, 
evaluation action, and history action 
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The actions, both global and local, are written by the user in a semantically based structure editor using hierarchical 
dialogues. There is no typing of text, simply selection of pop-up menu items. The system makes sure that the 
menu items presented to the user are syntactically and semantically appropriate. Thus the LingWorlds authoring 
system differs from Programming by Rehearsal in that LingWorids 1) does not provide for writing methods by 
^Svatching" the user, but 2) does provide a high-level, semantically appropriate language for constmction of actions 
rather than require the user to program in the underlying language in which the system was implemented. In Figure 
9, the author is expanding the template for an ^Tf-then-else" statement. 
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Figure 9. Programming an "If-then-elsc" Statement 
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Figure 10- Programming a **Says" Sutement 

Figure 10 shows, ih.e author conslructing an action in which the thing says tlie phrabe ''Good morning students." 

The tutor is also a thing. Actions can be called and written, using the same interface, for the tutor to produce 
appropriate direciion of the lesson. Direct selection of tutoring modes is also enabled. 

Using the Aulhoring System 

Without actions, things merely have Matic qualities. These qualities arc represented by, for example, the things 
graphic representation, its name, and its attributes. Having pivsented the various parts of **ie LingWorlds authoring 
system, we no\\ discuss the construction of things with four distinct kinds of functionality. We note that the 
functionality of things in LingW oilds reflects a set of levels of behavior inhereni lo intcrfavxs: behavior 1) aflecting 
only the thing itselt, 2) atVecting other things. 3) taking state into account, and 4) as a group or aggregation. 

Actions Ltmiled to Self: In a simple exploratory version ol the lifeboat lesson shown being built in Figure 
1, the lantern was set up so that it spoke its name when it was clicked. The attributes of the thing > ^ere set so that 
l-dick? was Tme and draggable? was True. The thing's 1 click action was constnicted to be flash self; say "The 
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lantern.'' Thus when, in running mode, when the user clicks the mouse once on the lanlem thing, it will flash 
itself and cause the phrase "The lantern" to be uttei^ The attributes and behavior of no other thing are affected. 

Actions AfTecting Other Things: Things can also have actions which affect each other. Let us suppose that 
the intent of the author is to have some things cause the lifeboat to flash whenever they are dragged. To accomplish 
this, the author sets the thing's landing-action? attribute to True and then constructs the thingfs landing-action local 
action as flash lifeboat As a result, any time the student stops dragging a thing with these characteristics, the 
lifeboat's image will flash. 

State-Based Actions: A more complex set of actions lakes into account the history of the user's interaction. 
Thus, for example, the author might wish to provide a history-based tutoring strategy. One way of doing this 
would be to create a new thing-attribute like 'dumber of wrong tries.* The tutor, then, in directing the snident to 
perform a task in the niicroworid could tailor its responses based on the value of this attribute for a particular thing. 
Thus if the value of the wrong-tries attribute for the lantern were zero, the tutor might simply repeat the instruction. 
If the value were sufficiently large, the tutor might demonstrate the requested task for the student. 

Group-Related Actions: Finally, we look at the behavior of explicitly grouped things. For example, the 
lifeboat thing consists of t\^^o things': the lifeboat and its "inside." which corresponds to the lifeboat's floor. The 
inside is used for determining if another thing is in fact "in" the lifeboat. Obviously, much more complex 
aggregations and behaviors can be obtained by suitable modification of the action of the constituent objects which 
compose the group. The important thing to note here is that the objects actions directly encode their interface 
functions. In a sense, LingWorids objects represent a "deconstruction" of the usual interface features. That is, the 
functional limitations of stereotypical interface components !'.:.\ ^ i Jcn removed; in their place arc the members ofihc 
basic function set of LingWorids prototypical objects. From these elements, new microworlds arc constructed. 

Implementation 

LingWorids is currently implemented flilly on a Macintosh II. The Macintosh version is written in CommonLisp. 
The movies and digitized sound are files created by existing commercial products to which we had to program some 
conversion software to load them into the Macintosh resources. All the actual code for the interactors and tutor 
objects are also loaded as resources. This gives us a "declarative" feel to the object-oriented code and reduces the 
complexity of loading in the program as part of the object-oriented environment. 



Evaluation 

Our informal evaluation of the system indicates that the average time to prototype a microworid is about several 
oiders of magnitude less the time taken by Pascal programmers on the Mac (days rather than months). We have yet 
to extensively test the system with more formal evaluation studies, but those are plan^ned for the future. Since the 
system is actually quite a powerful system for creating any instructional software, a version of the system was 
recently requested and sent to Yale University for use in building instructional software for mathematics tutoring. 

We have built both an English and Japanese version of Provisioning the UfcboaU with all the expected savings in 
programming. We built the exploratory version first, and then added the directed tutor version. A game version ha^ 
not been programmed, but would be trivial. Since interactors and tutors can be specialized for any particular 
microworid. most of the code is inherited and reusable. The Japanese version of the Lifeboat \s virtually identical to 
the English except for the addition of animacy features required for speech synthesis. Instantiation allows easy copies 
of objects with the minimum of programming effort. We have also built a microworid called FJadand, in which the 
student is taught geometric shapes, colors, size and spatial relations. This is a directed tutor version, based on 
extensive protocols with human tutors. The implementation of Flatland took one day, since almost ail of the code 
was reusable from the liTeftoaf problem. We have designed several other microworlcb of various types, as described 
above. 
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F. Summary and Conclusions 

In this report we have attempted to describe the principal outcomes and problems associated with our project to build 
an interesting and effective computer-based second language tutor, LingWorlds. Over the course of the project, we 
found that the basic problems of interest to us in creating this system increased in complexity and scope. However, 
the end result of the project is the creation of a ICAI system that teachers will be able to use coupled with important 
new insights into the nature of language learning and teaching. 
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